Skip to content

[feat](function) Add new function of strip_null_value #57386

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Nov 4, 2025
Merged

[feat](function) Add new function of strip_null_value #57386

merged 1 commit into from
Nov 4, 2025

Conversation

@mrhhsg
Copy link
Member

@mrhhsg mrhhsg commented Oct 27, 2025 •
edited
Loading

What problem does this PR solve?

Doc: apache/doris-website#3040

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

hello-stephen commented Oct 27, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@mrhhsg mrhhsg marked this pull request as draft October 27, 2025 15:11
@mrhhsg
Copy link
Member Author

mrhhsg commented Oct 27, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Oct 27, 2025

TPC-DS: Total hot run time: 190633 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b1c5989782a209a1f4056f75a949fe4d2bdd03fc, data reload: false

query1	1075	444	414	414
query2	6573	1703	1721	1703
query3	6763	224	224	224
query4	26768	23794	23134	23134
query5	5076	664	491	491
query6	350	256	239	239
query7	4653	534	302	302
query8	322	284	270	270
query9	8740	2655	2684	2655
query10	554	382	306	306
query11	15410	15146	14879	14879
query12	192	120	118	118
query13	1706	578	452	452
query14	12281	9401	9387	9387
query15	267	193	181	181
query16	7908	687	521	521
query17	1648	833	753	753
query18	2269	444	407	407
query19	287	233	202	202
query20	143	137	142	137
query21	230	148	132	132
query22	4852	4740	4534	4534
query23	35320	33732	33660	33660
query24	8369	2548	2599	2548
query25	601	560	499	499
query26	1286	309	164	164
query27	2737	568	379	379
query28	4693	2303	2256	2256
query29	924	658	515	515
query30	328	264	202	202
query31	947	838	801	801
query32	89	85	69	69
query33	591	411	347	347
query34	857	968	542	542
query35	899	875	792	792
query36	974	1153	936	936
query37	141	127	92	92
query38	3740	3912	3693	3693
query39	1505	1536	1491	1491
query40	236	136	121	121
query41	72	61	59	59
query42	130	112	108	108
query43	495	531	475	475
query44	1263	755	741	741
query45	180	186	181	181
query46	962	1043	633	633
query47	1739	1790	1706	1706
query48	410	436	324	324
query49	764	545	433	433
query50	694	732	408	408
query51	3855	4036	3841	3841
query52	110	111	103	103
query53	241	274	198	198
query54	616	600	531	531
query55	91	85	90	85
query56	315	338	304	304
query57	1168	1206	1116	1116
query58	294	281	277	277
query59	2547	2574	2475	2475
query60	363	355	357	355
query61	174	160	157	157
query62	794	768	680	680
query63	242	199	190	190
query64	4503	1185	844	844
query65	4057	3936	3973	3936
query66	1094	439	327	327
query67	15547	15309	15024	15024
query68	8514	911	610	610
query69	503	336	285	285
query70	1395	1257	1250	1250
query71	512	353	310	310
query72	5549	4946	4888	4888
query73	717	603	374	374
query74	9167	9005	8736	8736
query75	4012	3381	2869	2869
query76	3777	1222	738	738
query77	822	408	301	301
query78	9572	9804	8958	8958
query79	2388	837	601	601
query80	665	578	514	514
query81	494	263	234	234
query82	443	162	133	133
query83	293	273	256	256
query84	301	116	93	93
query85	898	489	441	441
query86	338	313	314	313
query87	3780	3873	3711	3711
query88	3245	2289	2300	2289
query89	416	334	296	296
query90	2064	223	214	214
query91	166	173	141	141
query92	86	68	69	68
query93	1170	1026	644	644
query94	692	453	340	340
query95	403	327	302	302
query96	508	626	285	285
query97	2963	2997	2869	2869
query98	247	227	208	208
query99	1453	1412	1303	1303
Total cold run time: 282577 ms
Total hot run time: 190633 ms

@doris-robot
Copy link

doris-robot commented Oct 27, 2025

ClickBench: Total hot run time: 27.91 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b1c5989782a209a1f4056f75a949fe4d2bdd03fc, data reload: false

query1	0.05	0.05	0.06
query2	0.09	0.05	0.05
query3	0.25	0.09	0.08
query4	1.62	0.12	0.12
query5	0.27	0.27	0.25
query6	1.15	0.66	0.65
query7	0.02	0.03	0.03
query8	0.06	0.05	0.05
query9	0.62	0.52	0.51
query10	0.58	0.58	0.58
query11	0.16	0.11	0.12
query12	0.18	0.13	0.12
query13	0.63	0.61	0.61
query14	1.01	1.01	1.00
query15	0.85	0.85	0.85
query16	0.41	0.40	0.39
query17	1.02	1.05	1.04
query18	0.22	0.20	0.20
query19	1.97	1.84	1.90
query20	0.02	0.01	0.02
query21	15.47	0.20	0.12
query22	5.04	0.07	0.05
query23	15.66	0.26	0.09
query24	2.17	1.43	0.59
query25	0.08	0.07	0.06
query26	0.15	0.13	0.13
query27	0.07	0.06	0.06
query28	4.37	1.13	0.96
query29	12.55	3.97	3.24
query30	0.29	0.14	0.12
query31	2.82	0.61	0.39
query32	3.24	0.55	0.47
query33	3.01	3.03	3.11
query34	15.76	5.09	4.58
query35	4.60	4.60	4.61
query36	0.69	0.51	0.49
query37	0.10	0.07	0.06
query38	0.06	0.04	0.04
query39	0.04	0.03	0.03
query40	0.17	0.15	0.15
query41	0.09	0.04	0.04
query42	0.03	0.03	0.03
query43	0.05	0.04	0.04
Total cold run time: 97.69 s
Total hot run time: 27.91 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Oct 27, 2025

FE Regression Coverage Report

Increment line coverage 91.67% (11/12) 🎉
Increment coverage report
Complete coverage report

@doris-robot
Copy link

doris-robot commented Oct 28, 2025

BE UT Coverage Report

Increment line coverage 7.69% (4/52) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.76% (18058/34228)
Line Coverage 37.99% (163692/430917)
Region Coverage 32.39% (124869/385487)
Branch Coverage 33.74% (54560/161728)

@hello-stephen
Copy link
Contributor

hello-stephen commented Oct 28, 2025

BE Regression && UT Coverage Report

Increment line coverage 90.38% (47/52) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 71.36% (23930/33533)
Line Coverage 57.77% (248803/430699)
Region Coverage 52.95% (206603/390191)
Branch Coverage 54.54% (88620/162493)

@mrhhsg mrhhsg marked this pull request as ready for review November 4, 2025 03:32
@mrhhsg
Copy link
Member Author

mrhhsg commented Nov 4, 2025

run buildall

@github-actions
Copy link
Contributor

github-actions bot commented Nov 4, 2025

PR approved by anyone and no changes requested.

@doris-robot
Copy link

doris-robot commented Nov 4, 2025

TPC-DS: Total hot run time: 189823 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b1c5989782a209a1f4056f75a949fe4d2bdd03fc, data reload: false

query1	1044	413	408	408
query2	6550	1739	1743	1739
query3	6754	231	221	221
query4	26477	23322	23006	23006
query5	4405	650	472	472
query6	351	256	254	254
query7	4651	508	305	305
query8	324	278	253	253
query9	8732	2605	2557	2557
query10	508	356	296	296
query11	15753	15119	14885	14885
query12	187	125	119	119
query13	1739	571	440	440
query14	10532	9448	9295	9295
query15	227	183	188	183
query16	7231	673	517	517
query17	1242	742	619	619
query18	1996	418	322	322
query19	213	206	173	173
query20	132	132	125	125
query21	218	132	125	125
query22	4636	4851	4675	4675
query23	34768	34055	33914	33914
query24	8652	2507	2489	2489
query25	685	591	501	501
query26	1350	283	171	171
query27	2930	529	368	368
query28	4423	2302	2244	2244
query29	920	672	536	536
query30	317	243	205	205
query31	940	848	764	764
query32	89	92	80	80
query33	749	467	330	330
query34	827	878	535	535
query35	857	868	781	781
query36	966	988	912	912
query37	128	123	91	91
query38	3722	3721	3598	3598
query39	1547	1486	1449	1449
query40	227	127	114	114
query41	60	57	58	57
query42	123	112	115	112
query43	501	486	471	471
query44	1204	748	741	741
query45	180	174	176	174
query46	924	992	630	630
query47	1795	1788	1779	1779
query48	399	423	325	325
query49	763	515	400	400
query50	637	675	407	407
query51	3920	3915	3785	3785
query52	108	109	101	101
query53	234	260	190	190
query54	302	287	309	287
query55	89	89	80	80
query56	325	322	306	306
query57	1170	1189	1117	1117
query58	300	280	269	269
query59	2554	2642	2595	2595
query60	345	342	328	328
query61	163	153	153	153
query62	786	744	652	652
query63	228	192	196	192
query64	4605	1189	892	892
query65	4067	3987	3962	3962
query66	1176	431	338	338
query67	15681	15065	14976	14976
query68	8570	929	596	596
query69	503	313	283	283
query70	1266	1350	1256	1256
query71	525	333	316	316
query72	6033	5044	4997	4997
query73	707	606	357	357
query74	9354	8894	8783	8783
query75	4035	3364	2822	2822
query76	3830	1158	728	728
query77	814	401	308	308
query78	9508	9653	8941	8941
query79	2735	832	595	595
query80	694	556	507	507
query81	476	266	231	231
query82	447	161	133	133
query83	291	274	248	248
query84	302	123	92	92
query85	902	491	456	456
query86	335	302	288	288
query87	3744	3695	3631	3631
query88	3525	2208	2201	2201
query89	395	324	303	303
query90	2045	212	208	208
query91	173	167	137	137
query92	80	63	62	62
query93	1858	968	641	641
query94	694	414	353	353
query95	412	323	315	315
query96	487	563	270	270
query97	2958	3000	2880	2880
query98	246	212	215	212
query99	1454	1448	1283	1283
Total cold run time: 280036 ms
Total hot run time: 189823 ms

@doris-robot
Copy link

doris-robot commented Nov 4, 2025

ClickBench: Total hot run time: 27.66 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b1c5989782a209a1f4056f75a949fe4d2bdd03fc, data reload: false

query1	0.05	0.05	0.04
query2	0.08	0.05	0.05
query3	0.25	0.08	0.08
query4	1.61	0.11	0.11
query5	0.27	0.27	0.25
query6	1.18	0.63	0.64
query7	0.03	0.02	0.02
query8	0.05	0.04	0.05
query9	0.60	0.52	0.53
query10	0.58	0.58	0.58
query11	0.17	0.11	0.11
query12	0.15	0.12	0.11
query13	0.62	0.59	0.60
query14	1.01	1.00	0.99
query15	0.85	0.82	0.84
query16	0.40	0.38	0.38
query17	1.02	1.00	1.04
query18	0.22	0.23	0.20
query19	1.88	1.78	1.85
query20	0.02	0.01	0.01
query21	15.44	0.19	0.13
query22	4.95	0.07	0.04
query23	15.70	0.26	0.10
query24	3.11	0.75	1.07
query25	0.08	0.06	0.05
query26	0.14	0.12	0.12
query27	0.06	0.06	0.06
query28	5.77	1.13	0.93
query29	12.57	3.83	3.22
query30	0.28	0.14	0.11
query31	2.81	0.61	0.39
query32	3.24	0.56	0.48
query33	3.02	3.03	3.01
query34	15.93	5.20	4.53
query35	4.65	4.57	4.62
query36	0.71	0.50	0.49
query37	0.10	0.07	0.08
query38	0.06	0.04	0.04
query39	0.04	0.03	0.03
query40	0.17	0.15	0.14
query41	0.09	0.03	0.03
query42	0.04	0.03	0.03
query43	0.05	0.04	0.03
Total cold run time: 100.05 s
Total hot run time: 27.66 s

@doris-robot
Copy link

doris-robot commented Nov 4, 2025

BE UT Coverage Report

Increment line coverage 7.69% (4/52) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.73% (18083/34294)
Line Coverage 38.05% (164469/432292)
Region Coverage 32.37% (125376/387298)
Branch Coverage 33.80% (54899/162402)

@hello-stephen
Copy link
Contributor

hello-stephen commented Nov 4, 2025

BE Regression && UT Coverage Report

Increment line coverage 90.38% (47/52) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 71.30% (24039/33713)
Line Coverage 57.66% (249677/433037)
Region Coverage 52.64% (206812/392894)
Branch Coverage 54.35% (88865/163511)

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 4, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Nov 4, 2025

PR approved by at least one committer and no changes requested.

@yiguolei yiguolei merged commit a22f24a into apache:master Nov 4, 2025
27 of 29 checks passed
github-actions bot pushed a commit that referenced this pull request Nov 4, 2025
@mrhhsg
[feat](function) Add new function of strip_null_value (#57386)
0d68601 
### What problem does this PR solve?

Doc: apache/doris-website#3040

Related PR: #xxx

Problem Summary:

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
@github-actions github-actions bot mentioned this pull request Nov 4, 2025
Merged
@mrhhsg mrhhsg deleted the add_strip branch November 4, 2025 09:25
yiguolei pushed a commit that referenced this pull request Nov 4, 2025
@github-actions @mrhhsg
branch-4.0: [feat](function) Add new function of strip_null_value #57386
a85f346 
 (#57689)

Cherry-picked from #57386

Co-authored-by: Jerry Hu <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@BiteTheDDDDt BiteTheDDDDt BiteTheDDDDt approved these changes

+1 more reviewer

@Mryange Mryange Mryange approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/4.0.1-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@mrhhsg @hello-stephen @doris-robot @BiteTheDDDDt @Mryange @yiguolei